Structure and Efficiency of Computer Programs

Much fundamental research in computer science is driven by two complementary notions of beauty in programming, that arising from the structure of a program and that arising from the efficiency of an algorithm. As Lewin famously said, “there’s nothing so practical as good theory” [Marrow, 1984]. Decades of experience has borne out the remarkable efficacy of theory in improving the practice of programming. Programming language theory is the study of the structural aspects of programming. The central notion is that of compositionality, the construction of a program by composition of separable parts. Results are formulated in terms of programming languages defined by type systems, which mediate composition of components, and semantics, which determines the execution behavior of those programs. Notable theorems include the parametricity theorem, which provides the mathematical foundation for the informal concept of data abstraction. Algorithm theory is the study of the efficiency aspects of programming. The central notion is of asymptotic analysis of the time and space usage of a program, expressed in terms of a size measure or a probability distribution on the inputs. Results are formulated in terms of machine models of computation, such as TM’s or RAM’s, which define the basic steps of an algorithm and their exceution cost, perhaps relative to parameters such as the number of processors on a PRAM. Algorithms are expressed in “high-level assembler”, a simple imperative programming language, for which the translation to the “official” machine model is readily understood. Often algorithms are expressed in terms of explicit representations of data structures using “pointers” and “words” to manage storage. By and large the algorithms and programming languages communities operate in isolation from one another. Programming languages researchers focus on the practicalities of building large software systems, and are seldom concerned about efficiency. Algorithms researchers focus on the efficiency of core programming techniques, and are seldom concerned about composition. (These are, of course, caricatures of reality, but it seems to me that they contain an element of truth.)